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Abstract 

In today’s terminal operations, controller workload increases and throughput decreases when 
fixed standard terminal arrival routes (STARs) are impacted by storms. To circumvent this operational 
constraint, Krozel, Penny, Prete, and Mitchell (2004) proposed to use automation to dynamically adapt 
arrival and departure routing based on weather predictions. The present study examined this proposal 
in the context of a NextGen trajectory-based operation concept, focusing on the acceptability of this 
proposal to both pilots and controllers, as well as its effect on the controllers’ ability to manage traffic 
flows. 

Six controllers and twelve transport pilots participated in a human-in-the-loop simulation of arrival 
operations into Louisville International Airport with interval management requirements. Three types of 
routing structures were used: Static STARs (similar to current routing, which require the trajectories of 
individual aircraft to be modified to avoid the weather), Dynamic routing (automated adaptive routing 
around weather), and Dynamic Adjusted routing (automated adaptive routing around weather with 
aircraft entry time adjusted to account for differences in route length). Spacing Responsibility, whether 
responsibility for interval management resided with the controllers (as today), or resided with the pilot 
(who used a flight deck based automated spacing algorithm), was also manipulated. We collected 
subjective workload and acceptability ratings at the end of each trial. Participants also provided additional 
ratings and comments in a debrief questionnaire administered at the end of the simulation. Task 
decisions and behaviors as well as verbal communications were also recorded. 

Dynamic routing as a whole was rated superior to static routing, especially by pilots, both in 
terms of workload reduction and flight path safety. Controllers also expressed a clear preference for 
dynamic routing. However, a downside of using dynamic routing was that the paths flown in the dynamic 
conditions tended to be somewhat longer than the paths flown in the static condition. 


Introduction 

Today, fixed routing structures such as standard terminal arrival routes (STARS) can become 
blocked by storms cutting across them. This leads to significant flow reductions as pilots request vectors 
around these blockages and controllers work to keep up with these requests. This, in turn, causes 
back-ups into the en route airspace, with accompanying fuel costs and delays. The Next Generation 
Air Transportation System (NextGen), seeks to transform the management of the National Airspace 
System (NAS), with one specific goal being to reduce these weather related delays (see JPDO, 2010). 
One proposal to accomplish this is to use automation, in combination with weather predictions, to adapt 
the arrival routing at regular intervals so that it bypasses weather systems (Krozel, Penny, Prete, & 
Mitchell, 2004). This could offload the problem of finding and implementing a clear path from the pilot 
and controller, thus reducing required communications and workload. However, the controller will then 
need to manage traffic on unfamiliar routes, which may be as difficult as simply managing spacing and 
separations without dynamic routing. A primary question for this study is whether dynamic routing is more 
acceptable to controllers and pilots than static routing. We have not looked at the quality of the weather 
avoidance algorithm itself, as we assume that this will improve over the years needed to implement such 
a system. Therefore, in this initial test, we provided perfect weather prediction and did not use weather 
patterns through which the algorithm could not find a clear path. 

Method 


Participants 
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Twelve commercial airline pilots with glass cockpit experience and six retired controllers were 
recruited for this simulation. Pilots were divided into two person crews with the more experienced pilot 
chosen as the captain. Confederate pseudo-pilots were used to fly additional background aircraft. 

Equipment 

Pilot stations. Low fidelity desktop systems were used to simulate two dual-pilot flight decks 
while a mid-fidelity fixed based simulator was used to simulate a third flight deck. All simulations used 
the Multi Aircraft Simulation system (MACS; Prevot et al., 2006) to emulate Boeing transport aircraft. All 
participant aircraft were equipped as high-end present day flight decks including, in particular, FANS and 
airborne weather radar simulations. 

Controller stations. Air traffic control (ATC) station simulators emulating an advanced prototype 
traffic display based on the standard ATC Display System Replacement (DSR) were also implemented 
using MACS, and displayed on a 30” high resolution monitor. The DSR was equipped with advanced 
conflict detection and resolution tools coupled with a graphical route planning tool and also displayed 
dynamic NexRad weather. Controllers were also provided with a schedule window that allowed them 
to see the time each plane was predicted to cross the TRACON meter fix. See Prevot et al. (2006) for a 
complete description of the DSR capabilities, and Battiste, Lawton, Lachter, Brandt, & Johnson, (2012) for 
further description of the controller procedures used in this study. 

Collection of Real Time Workload. In addition to the equipment described above, each 
participant also had a separate touch-screen computer used to collect real time workload ratings. 

Scenarios 

The dual-pilot and confederate flight decks were part of an arrival stream headed for Louisville 
International Airport (SDF), reaching top of descent shortly after entering the final high altitude en 
route sector. These arrivals flew west to east through the sector and negotiated a storm front near the 
center of the sector. In addition there was IFR departure traffic that used FANS datacom to request 
deviations around weather, and crossing en route VFR traffic that was pre-deviated to fly appropriately 
with respect to the weather. Each scenario lasted approximately 50 minutes, and focused exclusively 
on the en route arrival up to the meter fix. Each plane took about 20 minutes to fly from a point prior to 
the top of descent to the meter fix. Controllers worked the entire scenario which was followed by a brief 
post trial questionnaire about the scenario and a break before the next scenario. Within each scenario 
the participant pilots flew two aircraft. They successively flew the 3rd and 15th aircraft in the stream of 
arrivals, filling out a brief post trial questionnaire after each flight. 

Procedures 

For both of these options we presume the use of trajectory based operations (TBO), a 
fundamental part of NextGen, in which complete flight plan modifications are exchanged between aircraft 
and air traffic control. All dual-pilot flight decks (those not piloted by confederates) were arrival aircraft 
headed for Louisville International Airport (SDF), reaching top of descent shortly after entering the sector. 
Flight decks flew west to east through the sector and negotiated a storm front near the center of the 
sector. 

Each arrival entered the scenario 130 ± 10 seconds behind the previous aircraft (constrained so 
that every three planes had an average delay of 130 seconds); and was assigned an initial flight level (FL 
330, FL 350, FL 370, FL 390). All arrivals started the scenario with the en route arrival (CBSKT) loaded 
in their flight management system. For arrival in the dynamic routing trials the arrival routing was pre- 
deviated for weather. Thus, if the crew was comfortable with the route relative to the weather, no further 
deviation were needed. In the static routing trials the route was not pre-deviated, see Figure 1, crews in 
this condition were required to use their onboard tools to create and datalink a weather free route to ATC 
for approval. Departure traffic leaving SDF used FANS datacom to request deviations around weather 
and VFR en route traffic was created and pre-deviated to fly appropriately with respect to each weather 
patterns. 

Experimental Design 

Two independent fixed factors were manipulated: Tree Type and Spacing Responsibility. 
In addition there were three random factors, Aircraft Crew, Controller and Scenario. There were three 
levels of Tree Type: Static, Dynamic and Dynamic Adjusted. The static tree is a control condition where, 
much like today, the controller is required to respond to pilot deviation requests. The left side of Figure 1 
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shows the static tree. It is a symmetrical tree with four potential entry points, each equidistant from two 
downstream merge points and from the meter fix. Aircraft enter the tree at any of these points temporally 
spaced so they would reach the meter fix appropriately spaced if they followed the static tree. The 
Dynamic and Dynamic Adjusted conditions both used Metron’s simple tree planner algorithm with fixed 
quadrant size, fixed RNP, and a single TRACON entry meter fix (Prete, Krozel, Mitchell, Kim & Zou, 
2008). As can be seen in Figure 1 , the tree planner warps the routing, with the goal of a route that will 
allow aircraft to avoid the weather while in the tree. The tree adapts to the predicted weather every 15 
minutes so the controller will have as many as three different simultaneous tree structures. The difference 
between the Dynamic and Dynamic Adjusted conditions arises from the way aircraft are initially spaced 
as they enter the scenario. This factor was introduced because creating dynamic trees has interesting 
consequences for arrival spacing. Due to the warping of routes in a dynamic tree, and the fact a 
preceding aircraft may enter on any of the four entry points, an entering plane might have a very different 
distance to fly to the meter fix than the plane preceding it, making the spacing problem at merge points 
and the meter fix more difficult. In principle, under TBO such path stretches need not cause problems. 
Aircraft could know of weather deviations for the plane entering the tree just ahead of them and adjust 
their speed to arrive at the entry points appropriately spaced. As a result, we implemented the dynamic 
trees in two ways: “Dynamic” and “Dynamic Adjusted.” In the Dynamic condition, aircraft arrive at the 
outer leaves of the tree at intervals that are unaffected by the path length to the meter fix. In the Dynamic 
Adjusted condition, aircraft on shorter paths are delayed relative to those on longer paths. Because the 
difference between Dynamic and Dynamic Adjusted was subtle, participants were only informed as to 
whether the condition was Dynamic to avoid biasing them. 



Static STAR 


Dynamic STAR 


Figure 1 : Examples of Static and Dynamic tree structures, with weather. Notice how the STAR is 
adjusted in the Dynamic tree to avoid the weather. 

Because dynamic routing also affects the spacing solution, we wanted to examine these routing 
options in the context of how interval management is handled. There were two Spacing Responsibility 
conditions: Controllers Responsible and Pilots Responsible. In the Controllers Responsible condition, 
controllers managed spacing between aircraft much as they do today. In the Pilot Responsible condition, 
automation generated speed guidance to achieve the desired spacing goal of 130 seconds at CBSKT. 
Unless the pilots chose to intervene, the spacing algorithm commanded speed was coupled with the 
autopilot and did not require manual Mode Control Panel speed inputs (see Battiste et al, 2012). 

Results 
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Real-time subjective workload probe ratings were collected during each trial, while retrospective 
subjective workload and acceptability ratings were collected at the end of each trial. In addition, at the end 
of the simulation, study questionnaires were administered to all of the participants and additional ratings 
and comments gathered. Pilot and controller performance data were also collected. Here we discuss the 
distance flown. For results pertaining to interval management, frequency of communication, and other 
additional measures of performance, see Battiste et al. (2012). 

Workload Ratings 

Pilots and controllers rated their workload on a 1 Low - 5 Typical - 9 High scale at three-minute 
intervals throughout each trial. In the controller ratings there was no main effect of Tree Type on controller 
workload ratings collected during the trials (p = .26). Controllers rated real-time workload close to typical 
for the Static ( M = 5.10), Dynamic ( M = 4.87) and Dynamic Adjusted (M = 4.92) routing. There was, 
however, a main effect of Spacing Responsibility on controller workload ratings, F(1 ,5) = 11 .66, p < .05. 
Controllers reported higher workload throughout the trial during Controller Managed Spacing (M = 5.20) 
compared to Pilot Managed Spacing (M = 4.73) trials. No interaction was found (p = .93). 

In the pilot ratings, there was a significant main effect of Tree Type on pilot real-time workload 
ratings, F(1.24,13.65) = 6.22, p < .05. Pilots reported greater workload with Static ( M = 3.23) routing 
compared to Dynamic Adjusted ( M = 2.84, p = .058) routing. Dynamic (M = 2.89) routing was not 
significantly different from other routing. However, there was no main effect of Spacing Responsibility 
on pilot workload ratings, p = .23. Pilots reported similar workload levels during the Controller Managed 
Spacing ( M = 2.96) and Pilot Managed Spacing ( M = 3.03, p = .23) trials. No interaction was found (p 
= . 20 ). 

Pilots and controllers were also asked to rate their workload at the end of each trial. These ratings 
generally followed the real time ratings with controllers rating the controller spacing condition higher and 
little difference between tree types and pilots rating the static tree higher with little difference between 
spacing conditions. Pilots and controllers were also given more detailed questionnaires after completing 
all runs. These too followed the pattern of controllers being affected primarily by the spacing condition 
while pilots were affected primarily by the tree type. However, there were two exceptions to this which 
may give some insight into the participants thinking. First, even though controllers rated their workload 
and the outcomes (e.g., trajectories of aircraft in their sectors) similarly across the tree types, when asked 
which tree type they preferred they overwhelming choose the dynamic trees (because participants were 
not informed about the two types of dynamic trees, they couldn’t rate them separately). Specifically, 
controllers were asked about their preference with managing aircraft. Overall, controllers preferred to 
manage aircraft using Dynamic ( M = 4.83) routing compared to Static ( M = 2.5) routing, F(1,4) = 22.27 , 
p < .01 . Controllers also somewhat agreed ( M = 4.17) that managing arrivals with Dynamic routing gave 
them more time and that with more practice or training, they could safely manage a higher traffic density 
(M = 4.33). 

Distance and Time Flown 

This analysis was based on the participant aircraft only. The length of the path flown was taken to 
be the sum of the path flown at the end of the trial and the remaining distance to the runway at that point. 

The distributions of distance flown in each condition is shown in Figure 2. The median distance 
flown was noticeably shorter for Static routing than in either of the dynamic routings (1 83.6 vs 1 86.7 and 
1 86.8 nm in the Static, Dynamic, and Dynamic Adjusted, respectively). Three ANOVAs were conducted 
with distance flown as the dependent measure. The effect of Tree Type was significant when analyzed by 
Controller (F(2,10) = 10.42, p < .01) and Scenario (F(2,34) = 4.82, p < .05) but only marginally significant 
when analyzed by Aircraft Crew (F(2,10) = 3.98, p = .054). 

The reason Static routing produced shorter paths is apparent if you look at the paths actually 
flown by the aircraft. These paths are shown in Figure 3 broken down by Tree Type. Note that many 
aircraft on Static routing did not deviate. In fact, 25 of the 72 experimental aircraft in that condition 
made no lateral deviations from the original STAR, while all aircraft in the Dynamic conditions made 
significant deviations due to the warping of the Static Star routes. This is because the concept underlying 
this simulation required that the path flown by aircraft in the Dynamic conditions be weather free for 
approximately 35 minutes (the path must be able to accept aircraft for 15 minutes and must remain clear 
of weather while aircraft are in the tree, about 20 minutes) while aircraft on Static routing only deviated 
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when weather impacted the path at the moment the aircraft passed. Note that no attempt was made to 
synchronize the weather with individual aircraft during scenario development. 
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Figure 2: “Tukey” boxplot (Tukey, 1973) of the distance flown to the runway by the experimental planes 
in each condition. For each condition the solid line indicates the median; the box indicates the inter- 
quartile range (IQR); the whiskers indicate the range excluding “outliers”; and the circles indicate outliers, 
points more than 1 .5 IQR from the box. The horizontal line indicates the distance flown (1 83 nm) on the 
static tree. 
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Figure 3. Paths of all experimental aircraft in each condition. Paths are transparent so that multiple 
aircraft taking the same path results in the path being darker. 

Summary and Discussion 

A large simulation like this creates a very large data set with many nuanced findings. Statistically, 
a few differences will be found to be significant when no real difference exists and some will not be found 
significant even when real differences do exist. It is important, therefore, to look at which differences show 
up repeatedly, across a variety of measures, and which show up rarely or not at all. In doing so, a few 
generalizations become clear: 

• Dynamic routing is rated superior to static, especially by pilots. 

• The two dynamic tree conditions (Dynamic and Dynamic Adjusted) are rated similarly 

We will briefly discuss each of these. 

Dynamic Routing vs Static Trees 

Pilots rated their workload lower in the dynamic routing conditions than they did in the static 
condition on every workload question except for the workload due to spacing (where it was roughly 
equivalent to the Dynamic). Workload for the Static Star condition was higher relative to the Dynamic 
condition for both overall and peak workload associated with avoiding weather, and relative to the 
Dynamic Adjusted condition for real-time workload (marginal), overall and peak flight (marginal). Pilots 
also felt safer in the Dynamic conditions than they did in the Static conditions. They rated the flight path 
flown least safe on the Static trials on the post trial questionnaire and they rated operations less safe in 
the Static conditions in the post simulation questionnaire. This is perhaps unsurprising since they came 
closer to the weather on the Static trials. Finally, on the post simulation questionnaire, pilots rated their 
ability to manage weather deviations higher in the Dynamic conditions than in the Static. 

Interestingly, controllers did not rate the Static and Dynamic conditions differently on any of the 
workload or safety questions. Yet, controllers agreed with the statement that it was easier to manage 
traffic in the Dynamic condition than the Static condition and they expressed a clear preference for 
Dynamic when asked if they prefer to manage traffic using the Static or Dynamic trees in the post 
simulation questionnaire. 

Dynamic vs Dynamic Adjusted 

It is interesting to note that few differences were found between the Dynamic and Dynamic 
Adjusted conditions in respect to subjective responses. While there are measures on which Dynamic 
Adjusted seems to have some advantage over Dynamic (e.g., pilot ratings of their workload associated 
with spacing) there are several on which Dynamic trends better than Dynamic Adjusted (e.g., controller 
real-time ratings, controller ratings of workload associated with maintaining spacing and controller ratings 
of workload associated with spacing). Clearly there was a spacing problem to be solved in all conditions 
as the controllers found their workload increase noticeably when they were responsible for managing it. In 
the Dynamic condition, planes entered the scenario with much more variable spacing than in the Dynamic 
Adjusted scenario (about double the RMS error). However, it is possible that even in the Dynamic 
Adjusted condition, controllers only have to make slight adjustments to the spacing of most planes or that 
larger adjustments are not more difficult than smaller ones. 
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